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Overview 


Evidence shows that financial aid increases college enrollment. For many students at low-cost 
community colleges, this aid is intended to cover more than tuition and fees; after those are paid, the 
remainder is paid out, or “refunded,” to students to help with their living expenses while they are 
enrolled in school. Often, however, the total amount of aid does not come close to covering the cost 
of attendance for full-time students. Many still struggle to find the means to support their studies and 
pay their bills, and the majority of students enrolled at two-year public institutions report feeling 
financial stress related to paying for school. 

Most colleges distribute financial aid refund amounts to students in one or two lump sums during the 
term. Aid Like A Paycheck is a study of an alternative approach, in which financial aid refunds are 
disbursed biweekly, with the goal of helping students stretch their financial aid (including federal 
Pell Grants, state aid, and loans) to cover expenses throughout the term. MDRC is conducting a 
mixed-methods study of biweekly disbursements at two community colleges in the metropolitan 
area of Houston, Texas. The study includes qualitative research on the program’s implementation 
and a randomized controlled trial to rigorously estimate the impacts of the policy on students’ 
academic and financial outcomes. 

Results at the midpoint of the evaluation present a mixed picture: 

• The colleges were able to disburse aid biweekly as intended, but communications about the 
policy and about financial aid in general were often unclear to students. 

• ft appears that students assigned to receive biweekly disbursements were more likely than those 
receiving a standard lump sum to feel that their finances caused significant stress at the start of 
the term. By the end of the semester, students in the two groups reported comparable levels of 
financial stress. 

• Biweekly disbursements reduced students’ use of federal loans and debt to the college after one 
semester, without reducing the overall aid they received. 

• On average, there is little consistent evidence of biweekly disbursements improving students’ 
key academic outcomes. However, at one college, students in the biweekly group experienced a 
6 percentage point increase in enrollment in the second semester of school. 

• Although the policy raised the possibility of cost savings for the colleges and the federal 
government, there is little evidence of such savings so far. 

These interim findings should be viewed with caution, but colleges interested in moving to biweekly 
disbursements may find useful lessons from the early implemented, as discussed in this report. 

In 2018 MDRC will present final results based on further research, including additional students and 
longer follow-up, to better understand the impacts of biweekly disbursements at the two Houston- 
area colleges. The final report will also include a deeper look at the program’s implementation at a 
third community college district, where biweekly disbursements of financial aid refunds recently 
became the standard policy for all students. 
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Preface 


In recent years, college leaders and policymakers have paid growing attention to the ways that 
students’ finances affect their academic success. Research demonstrates that financial aid helps 
students enroll and succeed in college. Yet even with financial aid, many low-income students 
are compelled to work while attending college, and many struggle to afford rent, food, and 
transportation to and from school. College financing, and student loans in particular, has also 
garnered public attention, as policymakers consider ways to maintain or improve students’ 
access to higher education while limiting potentially harmful debt, which some students carry 
for years. 

Aid Like A Paycheck is MDRC’s study of a new policy intended to improve the distri¬ 
bution of financial aid. Traditionally, colleges disburse student aid at the start of the semester, 
first covering the costs of tuition, fees, books, and supplies, then paying any remaining aid to the 
student in one or two lump sums near the start of the term. Students are expected to use these 
refunds to support their studies throughout the term. If they withdraw from classes, they may be 
required to return part of their aid to the college (and the college may be required to pay back 
these funds to the federal government). In Aid Like A Paycheck, by contrast, students receive 
their aid refunds — including Pell Grants, loans, and state aid — in smaller amounts, biweekly. 
The study is designed to answer several questions: Can incremental disbursements help students 
stretch their limited aid refunds throughout the semester? Do biweekly disbursements reduce 
the amount of students’ debt? Can this modest intervention increase students’ academic suc¬ 
cess? Or, conversely, does the policy unintentionally hurt students? 

Halfway through the study’s follow-up period, it appears that students are not harmed 
— and by some measures are helped — by biweekly disbursements of financial aid. While the 
impacts of the policy do not appear large, it is promising that a change in financial aid dis¬ 
bursement that does not include additional funds may offer benefits to students. This report 
provides details of these interim findings and sets the stage for a final report in 2018, which will 
include data on additional students with longer follow-up to better understand how colleges may 
be able to change financial aid disbursements to best support their students. 

Gordon L. Berlin 

President, MDRC 
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Executive Summary 


When community college students receive a financial aid refund after their tuition and fees have 
been paid, the amount of the refund may elicit a reaction like this: 

Students need more money for college, in my mind. It’s not enough for me. 1 
need more to pay for my courses, food, gas, and to pay all my bills like car and 
insurance. 

Yet at the same time, a lump sum refund may inspire caution: 

[I have] seen friends get huge sums of money — $2,000 to $3,000 — all at once 
and go ballistic. [I’d prefer to get it biweekly, because I] don’t like so much 
money just sitting there. 1 

Evidence shows that financial aid increases college enrollment and success. 2 For many 
students at low-cost community colleges, this aid is intended not only to cover tuition and fees, 
but also to help with their living expenses while they are enrolled in school. Often, however, the 
total amount of aid does not come close to covering the cost of attendance for full-time students. 
Many still struggle to find the means to support their studies and pay their bills, and the majority 
of students enrolled at two-year public institutions report feeling financial stress related to 
paying for school. 

In standard financial aid practice, colleges first apply students’ awards to tuition and 
fees, then often allow students to use the aid for book purchases at the start of the term, and 
finally disburse any remaining funds as a financial aid “refund.” Most colleges distribute this 


'Quotes are from students in focus groups and an online survey of students in the study. 

2 Susan Dynarski, “Does Aid Matter? Measuring the Effect of Student Aid on College Attendance and 
Completion,” American Economic Association 93, 1 (2003): 279-288; Benjamin L. Castleman and Bridget 
Terry Long, Looking Beyond Enrollment: The Causal Effect of Need-Based Grants on College Access, 
Persistence, and Graduation, Working Paper 19306 (Cambridge, MA: National Bureau of Economic 
Research, 2013). Also see Judith Scott-Clayton, Undergraduate Financial Aid in the United States (Cam¬ 
bridge, MA: American Academy of Arts and Sciences, 2017). 

’’The National Student Financial Wellness study in 2015 found that while 70 percent of all postsecond¬ 
ary students feel stress regarding their personal finances, students at two-year public institutions are more 
likely to worry about having enough to pay for school (Ohio State University, National Student Financial 
Wellness Study: Key Findings Report [Columbus: Ohio State University, 2015]). Letkiewicz and colleagues 
(2014) found that undergraduate students who felt significant amounts of financial stress were more likely to 
take longer than four years to complete their degree; see Jodi Letkiewicz, Hanna Lim, Stuart Heckman, 
Suzanne Bartholomae, Jonathan J. Fox, and Catherine P. Montalto, “The Path to Graduation: Factors 
Predicting On-Time Graduation Rates,” Journal of College Student Retention: Research, Theory’ & Practice 
16, 3 (2014): 351-371. 
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refund to students in one or two lump sums during the term. Anecdotal evidence suggests that 
many students who receive refunds use that aid to pay for substantial expenses, such as rent, car 
loans, or large credit card payments, and the lump sum sometimes runs out before the end of the 
semester. Furthermore, research has shown that unexpected spikes and fluctuations in income 
may interfere with budgeting and financial planning. 4 

Aid Like A Paycheck is a study of whether biweekly — as opposed to lump sum — fi¬ 
nancial aid disbursements can help students stretch their financial aid to cover expenses 
throughout the term and whether such a policy can improve students’ academic and financial 
outcomes. By evenly distributing a student’s existing aid, as depicted in Figure ES.l, rather than 
offering additional financial support, incremental disbursements are a modest intervention yet 
may have positive impacts. Disbursing aid refunds in biweekly increments is theorized to help 
students by means of mechanical changes in the timing and disbursements of aid as well as 
changes in messaging about how to budget the aid to last through the semester. 

In addition, by distributing aid as it is earned rather than in advance, biweekly dis¬ 
bursements may reduce debt that may be incurred because of later enrollment changes — 
students’ potential debt to the college or colleges’ potential debt to the Department of Educa¬ 
tion. The policy might thus provide financial benefits to students and cost savings to colleges. 

MDRC is conducting a mixed-methods study of the Aid Like A Paycheck policy at two 
community colleges in the metropolitan area of Houston, Texas: San Jacinto College and the 
Houston Community College System (HCC). HCC is the fifth largest community college 
system in the country, comprising six colleges and more than 15 campuses spread across the 
city and serving about 90,000 students per academic year. 5 San Jacinto has three campuses and 
serves about 40,000 students per academic year. 6 

Beginning in 2014, eligible students at these colleges were randomly assigned to either 
the program group, whose members received their financial aid refunds — including federal 
Pell Grants, loans, and other aid — in biweekly disbursements, or to the standard group (also 
known as the control group), whose members received their financial aid refunds under the 


Jonathan Morduch and Rachel Schneider, Spikes and Dips: How Income Uncertainty Affects House¬ 
holds (New York: U.S. Financial Diaries Project, 2013). 

5 Houston Community College, Houston Community College 2015-2016 Fact Book, revised September 
30, 2016; Integrated Postsecondary Education Data System (IPEDS), U.S. Department of Education, 
Institute of Education Sciences, National Center for Education Statistics (https://nces.ed.gov/ipeds). 

6 IPEDS. 
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Figure ES.l 

Two Methods of Disbursing Aid 
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colleges’ standard lump sum aid disbursement policy. 7 This report presents interim findings 
based on the experiences of over 6,000 students enrolled over three semesters — with one to 
two semesters of follow-up — to estimate whether students assigned to receive biweekly 
disbursements have different financial and academic outcomes from those of students assigned 
to receive their aid in the standard way. MDRC researchers also fielded a student survey, 
conducted small focus groups of students, and spoke with financial aid staff members and 
administrators at the colleges to understand their experiences, attitudes, and behaviors. 

Key Findings 

Results at the midpoint of the evaluation present a mixed picture: 

• The colleges were able to implement the mechanics of biweekly dis¬ 
bursements as intended. In their first semester, most students in the pro¬ 
gram group received six to eight refunds, averaging between $300 and $500 
per refund, and most students in the standard group received just one or two 
refunds, averaging between $1,500 and $2,000 per refund. 

• College communications about the policy and about financial aid in gen¬ 
eral were often unclear to students. The colleges worked to improve their 
program communications over the first two years of implementing Aid Like 
A Paycheck. 

• Students who receive financial aid refunds after payment of tuition and 
fees are by definition some of the lowest-income students, and not sur¬ 
prisingly, many students in the study reported struggling financially. 

Regardless of whether they received their aid in a lump sum or biweekly, 
many students reported using their financial aid refunds to pay for school 
supplies, transportation, and other living expenses, and about half the stu¬ 
dents who responded to the survey stated that they were working for pay 
while attending college. 

• It appears that students assigned to receive biweekly disbursements 
were more likely than those receiving a standard lump sum refund to 
feel that their finances caused significant stress at the start of the term. 

By the end of the semester, students in the two groups reported comparable 
levels of financial stress. 


7 Students in this study were selected through college records based on the criterion of having an ex¬ 
pected financial aid refund amount of $800 or more after tuition and fees were paid and allowing for book 
expenditures. 
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• Biweekly disbursements reduced the use of federal loans and students’ 
debt to the college after one semester. Students assigned to receive biweek¬ 
ly aid and those assigned to receive their aid in the standard way received the 
same total amount of financial aid, but students in the program group re¬ 
ceived an average of $82 less in loan disbursements and had an average of 
$24 less in debt to the college after their first semester. 

• On average, there is little consistent evidence of biweekly disbursements 
improving students’ key academic outcomes. However, at one college, 
students in the biweekly disbursement group experienced a 6 percentage 
point increase in enrollment in the second semester of school. 

• There is little evidence of the expected cost savings to the colleges from 
the policy of biweekly disbursements. This is not surprising, given the lack 
of substantial impacts on overall financial outcomes at this point in the study. 

Nor is there evidence of substantial cost impacts for the state or the federal 
government. 

Conclusion 

Many students struggle financially, and although biweekly disbursements of financial aid 
cannot change the fact that existing aid often does not cover the full costs of attendance, this 
policy still has the potential to provide academic and financial benefits to students. At the 
midpoint of the study, students assigned to receive their aid biweekly were not harmed academ¬ 
ically or financially by the policy, and on some measures were better off than those who 
received a standard lump sum refund. 

At the current point in the research, several key questions remain unanswered: Why are 
academic impacts evident at one college but not at the other? Will the small financial impacts of 
the program accumulate, suggesting there could be substantial financial benefits or savings in 
the long run? If the policy of biweekly disbursements of financial aid were to be applied as a 
college-wide norm, could it be implemented more efficiently and communicated more clearly to 
students? As long as these questions remain unanswered, other colleges should look at these 
findings with caution. Nevertheless, biweekly disbursements may be appealing to colleges 
wishing to be transparent in aligning the awarding of aid with continued enrollment, to signal to 
students and others that financial aid refunds are intended to support students’ expenses while in 
school. These colleges may find useful lessons from the early implementers, as discussed in the 
full report. 

In 2018, MDRC will present final results based on further research, including an addi¬ 
tional 2,500 students and up to three more semesters of follow-up, to better understand the 
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impacts of biweekly disbursements on students’ outcomes at the two Houston-area colleges. 
The final report will also include a deeper look at the program’s implementation at a separate 
small community college district (not included in this report’s quantitative findings) where 
biweekly disbursements of financial aid refunds became the standard policy for all students in 
the 2016-2017 academic year. 
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Introduction 

Fewer than 40 percent of community college students receive a degree within six years of 
enrollment; for low-income students, this percentage is even smaller. 1 Low-income community 
college students face a number of nonacademic obstacles to completing their college education, 
including unreliable transportation, job responsibilities, personal emergencies, and arranging for 
housing, food, and child care. 2 In addition to these challenges, financial stress can limit stu¬ 
dents’ success in postsecondary institutions, and the majority of students enrolled at two-year 
public institutions report feeling financial stress related to paying for school. 3 

Federal Pell Grants and loan programs are the two largest sources of economic aid for 
college students, with additional support from emergency aid, state aid, and other limited 
resources. 4 Evidence shows that financial aid increases college enrollment and success, and 
programs that offer funds in addition to Pell Grants and other federal and state aid, such as 
performance-based scholarships, have been shown to have modest positive effects on student 
outcomes. 5 However, the complexity of financial aid can actually add to students’ financial 
stress and may undermine aid programs’ effectiveness. Pell Grants require students to complete 
the Free Application for Federal Student Aid (FAFSA), a complex and lengthy process that has 
been shown to discourage low-income individuals from successfully applying for aid. 6 College 
staff members report that students are often unfamiliar with the financial aid award process, 
despite financial aid offices’ attempts to communicate with them. 7 And students are often 
unaware of the satisfactory academic progress requirements tied to eligibility for certain types 
of financial aid. Failure to meet academic standards can lead to a loss of Pell Grant funding, 
which may contribute to a reduction in college persistence and degree attainment. 8 


*Ma and Baum (2016); Ginder, Kelly-Reid, and Mann (2015); Juszkiewicz (2015). 

2 Matus-Grossman and Gooden (2002) discuss how child care issues, rocky personal relationships, and 
lack of employer support are common challenges among low-income students who are also parents. Over half 
the survey respondents in Goldrick-Rab, Broton, and Eisenberg (2015) had experienced at least marginal food 
insecurity in the last 30 days, and Wood, Harris, and Delgado (2016) note that about one-third of California 
community college students reported experiencing housing insecurity. 

3 Letkiewicz et al. (2014) find that undergraduate students who feel significant amounts of financial stress 
are more likely to take longer than four years to complete their degree. The National Student Financial 
Wellness Study (Ohio State University, 2015) found that while 70 percent of all postsecondary students feel 
stress regarding their personal finances, students at two-year public institutions are more likely to worry about 
having enough to pay for school. 

4 Dynarski and Scott-Clayton (2013). 

5 Dynarski (2003); Castleman and Long (2013); Patel, Richburg-Hayes, de la Campa, and Rudd (2013). 
Also see Scott-Clayton (2017). 

6 Bettinger, Long, Oreopoulos, and Sanbonmatsu (2012). 

7 Cochrane (2007). 

8 Schudde and Scott-Clayton (2014). 
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In standard financial aid practice, colleges first apply students’ awards to tuition and 
fees, then often allow students to use the aid for book purchases at the start of the tenn, and 
finally disburse any remaining funds as a financial aid “refund.” Most colleges distribute this 
refund to students in one or two lump sums during the tenn. This system of financial aid 
disbursement is common at colleges nationwide, but students are most likely to receive refunds 
in states or at colleges with low costs or with relatively generous state or institutional aid 
covering tuition and fees. Such institutions tend to have the highest portion of students with 
remaining financial aid available for refund. 9 

Aid Like A Paycheck is a study of whether biweekly — as opposed to lump sum — fi¬ 
nancial aid disbursements can help students stretch their financial aid to cover expenses 
throughout the tenn and whether such a policy can improve students’ academic and financial 
outcomes. 


Aid Like A Paycheck 

When community college students receive a financial aid refund after their tuition and fees have 
been paid, the amount of the refund may elicit a reaction like this: 

Students need more money for college, in my mind. It’s not enough for me. I 
need more to pay for my courses, food, gas, and to pay all my bills like car and 
insurance. 

Yet at the same time, a lump sum refund may inspire caution: 

[I have] seen friends get huge sums of money — $2,000 to $3,000 — all at once 
and go ballistic. [I’d prefer to get it biweekly, because I] don’t like so much 
money just sitting there. 10 

Anecdotal evidence suggests that many students who receive refunds use this financial 
aid to pay for substantial expenses, such as rent, car loans, or large credit card payments, and the 
lump sum sometimes runs out before the end of the semester. Furthermore, research has shown 
that unexpected spikes and fluctuations in income may interfere with budgeting and financial 
planning. 11 

In many policy areas, there is growing interest in innovative programs that work 
through existing funding and structures to improve participant outcomes. For example, a 
program offering periodic payments of the Earned Income Tax Credit (EITC) — which would 


9 Ware, Weissman, and McDermott (2013). 

1 ''Quotes are from students in focus groups and an online survey of students in the study. 
"Morduch and Schneider (2013). 
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seek to improve money management and financial health by changing the disbursement 
schedule of preexisting funds — has gained traction among researchers and the U.S. Congress 
in recent years. 12 Research has shown that low-income families often use EITC money dis¬ 
bursed in a lump sum for larger purchases rather than treating these funds as part of their regular 
income stream. 13 A 2015 study of the Chicago Periodic Payment Pilot found that this disburse¬ 
ment schedule is administratively feasible, is preferred by taxpayers, and has been shown to 
improve financial stability by matching the timing of the disbursement with the need of the 
individuals. 14 Likewise, several promising money management apps and products targeted to 
financially underserved populations seek to promote financial health through encouraging 
saving and mindful spending as well as cash-flow smoothing by nudging users to stretch limited 
funds over the course of a month. 15 

Aid Like A Paycheck differs from these interventions in many ways, but the policy 
takes a similar approach of innovating with existing resources to improve the academic and 
financial circumstances of community college students. By evenly distributing a student’s 
existing aid rather than offering additional financial support, Aid Like A Paycheck is a 
modest intervention, yet it may have positive impacts. Biweekly financial aid disbursements 
— including federal Pell Grants, loans, and other aid — are theorized to help students by 
means of mechanical changes in the timing and disbursements of aid as well as changes in 
messaging about budgeting their aid to last through the semester. In addition, by distributing 
aid as it is earned rather than in advance, biweekly disbursements may reduce debt, which is 
sometimes incurred when students reduce their course loads or withdraw from college. Such 
enrollment changes can result in student debt to the college or college debt to the Department 
of Education. 

MDRC is conducting an implementation and impact evaluation of the Aid Like A 
Paycheck policy at two community colleges in and around Houston, Texas: San Jacinto College 
(San Jacinto) and the Houston Community College System (HCC). This report presents 
findings based on the experiences of over 6,000 students enrolled over three semesters — with 
one to two semesters of follow-up — to estimate whether students randomly assigned to receive 
biweekly disbursements have different financial and academic outcomes from those of students 
assigned to receive their aid in the standard lump sum. Overall, the interim results present a 
mixed picture. The colleges were able to implement the mechanics of biweekly disbursements 
as intended; however, college communications about the policy and about financial aid in 


12 Holt (2015). 

13 Barrow and McGranahan (2000); Goodman-Bacon and McGranahan (2008). 
14 Bellisle and Marzahl (2015). 

15 Sneller (2016). 
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general were often unclear to students. It appears that biweekly disbursements did not relieve 
financial stress, but students assigned to receive their aid biweekly were not harmed academi¬ 
cally or financially by the policy, and on some measures were better off than those who re¬ 
ceived a standard lump sum refund. 


How Could Biweekly Disbursements Affect Student Outcomes? 

In contrast to the standard financial aid practice of disbursing financial aid refunds in one or 
two lump sums, colleges could divide students’ remaining aid into roughly equal incremental 
disbursements throughout the term, as depicted in Figure 1. This policy of incremental 
disbursements fits within the Department of Education regulations that allow any college to 
“pay a student at such times and in such installments as it determines will best meet the 
student’s needs.” 16 

An early pilot evaluation in 2010 to 2013 at Mt. San Antonio College in California and 
Triton College in Illinois demonstrated that it is administratively possible to implement biweek¬ 
ly payments of financial aid refunds at community colleges; the model showed promise for 
positive impacts. 17 The Aid Like A Paycheck study began in 2014, with two large Houston-area 
community colleges disbursing financial aid biweekly to a randomly selected group of students, 
with the goal of helping students achieve greater academic success. 

The Aid Like A Paycheck model aims to assist students who may have difficulty man¬ 
aging relatively large lump sum refunds and who lack financial support when they need it. 
When students receive incremental, biweekly aid refunds, they may experience increased 
financial stability and reduced financial stress, which may enable them to stay in college, reduce 
their borrowing, and ultimately persist in college with less debt. Incremental refunds may also 
help reduce Returns to Title IV (known as “R2T4”), in which federal financial aid must be 
repaid when students withdraw from classes. (Box 1 explains that policy and its consequences 
in more detail.) 

Altering the timing of aid does not increase the total amount of aid available to low- 
income students. Biweekly disbursements are unlikely to change the fact that Pell Grants and 


16 This explicit permission to colleges to determine how federal financial aid is disbursed pertains to all 
grant and loan funds under Title IV of the Higher Education Act, which provides the majority of federal 
student aid. 

17 See Ware, Weissman, and McDermott (2013) for the origins of this evaluation and further description of 
the biweekly disbursement policy pilot, which was launched in 2009 by MDRC and the Institute for College 
Access & Success (TICAS). 
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Figure 1 

Two Methods of Disbursing Aid 
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Box 1 

Return to Title IV 

Title IV of the Higher Education Act, which provides the majority of federal student aid, 
includes a policy that requires the calculation of a Return to Title IV (commonly known as 
R2T4) when a student withdraws from all courses before completing 60 percent of the semes¬ 
ter. In these instances, colleges can be required to repay unearned aid funds to the Department 
of Education. In turn, colleges may attempt to recoup these funds from students and may bar 
students who do not repay them from future enrollment. Returns to Title IV are thus generally 
seen as a problem for both students and colleges. 

Financial aid administrators at the two study colleges discussed in this report explained that 
beyond the primary goal of helping students perform better in school, an important secondary 
benefit of implementing Aid Like A Paycheck was the potential for reductions in R2T4 as a 
result of aligning disbursement of aid more closely with the time that it is earned. 


other aid often do not come close to covering the cost of attendance for full-time students, and 
that students may still struggle to find the means to support their studies and ongoing living 
expenses. 18 The typical annual cost of attendance for a community college student is around 
$15,000, yet a full Pell Grant is only $5,775, suggesting that the refund amounts — whether 
paid in one lump sum per semester or biweekly throughout the tenn — may leave many 
students with unmet need. 19 Furthennore, the smaller biweekly disbursements could feel less 
substantial to students, leading them to feel additional financial stress, increase borrowing, or 
increase their work hours. 

In addition to the mechanical changes in the disbursements of aid discussed above, col¬ 
leges can change their messaging about financial aid; the combination is the basis for the theory 
of change shown in Figure 2. Communications from colleges that accompany biweekly dis¬ 
bursements could potentially alter students’ attitudes and behaviors about financial manage¬ 
ment, helping them plan how to use their aid to partially offset ongoing expenses such as rent, 
food, or child care over the course of the tenn. These communications could include email 
reminders from colleges about the financial aid disbursement schedule, conversations with 
financial aid or academic advisers, or financial literacy workshops in first-year orientation or 
student success courses. Together with ongoing financial support, explaining the system may 


18 Cochrane (2007). 

19 U.S. Department of Education Federal Pell Grant Program (CFDA 84.063). The maximum federal Pell 
Grant for the 2017-2018 award year will increase to $5,920 (https://fmancialaidtoolkit.ed.gov). The typical 
annual cost of attendance of a two-year public college was estimated at $15,286 by the College Board 
(https://bigfuture.collegeboard.org) at the time of publication. 
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Figure 2 

Aid Like A Paycheck Theory of Change 
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give students an incentive to stay in school — in order to continue receiving their aid and avoid 
having to repay it — and also encourage students to work less (particularly during finals) and 
study more, which could lead to better performance in classes. 

The myriad of influences and communications present at any college may limit the in¬ 
fluence that this one policy can have on students’ behaviors and attitudes. Many students also 
work for pay, receive housing or other support from family members, or have other means of 
sustaining themselves. Additionally, the aid being refunded is — for many students —just one 
part of their financial support. 20 But while the intervention of biweekly disbursements may be 
modest and only a part of students’ overall financial picture, the potential for short-term benefi¬ 
cial impacts outlined above may be sufficient to lead to increased persistence and greater credit 
accumulation, which ultimately may lead to increased rates of graduation or transfer. 

In addition to the potential impacts on students’ academic and financial outcomes, bi¬ 
weekly disbursements may provide financial benefits to colleges, particularly if the policy can 
reduce R2T4 or other student debt to the college. 21 The federal government could potentially 
reap financial benefits as well, and changes in student use of federal loans would affect the 
federal budget, although it is not clear whether a decrease in loans should be seen as a benefit or 
cost to the government. 


The Study Design 

Aid Like A Paycheck is MDRC’s mixed-methods study to test the theory of change described 
above. The study includes a randomized controlled trial, an evaluation design that is widely 
accepted to yield the most credible estimates of a program or intervention’s effects because it 
makes it possible to determine the counterfactual, or what would have happened in the absence 
of the intervention. In Aid Like A Paycheck, eligible students were randomly assigned either to 
the program group, whose members received financial aid under the new biweekly disburse¬ 
ment policy, or to the standard group (also known as the control group), whose members 
received financial aid under the colleges’ standard aid disbursement policy. Because of random 
assignment, members of the two groups can be expected to be alike in measurable traits (such as 
education and economic status) as well as other characteristics that are difficult to measure (like 


20 Researchers have raised concerns that federal financial aid remains a small amount of money in compar¬ 
ison to increases in the cost of college attendance, and about the often unstable availability of additional aid 
such as state grants, institutional grants, or work-study arrangements (Goldrick-Rab, 2013). 

2l The costs and burden of R2T4 fall primarily on colleges, which must pay back the federal government 
and are only sometimes able to recoup these funds from students. Other student debt to the college can result 
from students’ enrollment changes or factors such as parking tickets, library fines, student fees, and other 
charges which may be deducted from a student’s financial aid refund or billed to the student. 



motivation). When the two groups are followed over time, the differences in their outcomes 
provide a reliable estimate of the program’s effects, or impacts. 

The Aid Like A Paycheck study is designed to evaluate the implementation and impacts 
of biweekly disbursements at San Jacinto and HCC. 22 This report presents findings based on the 
experiences of more than 6,000 students enrolled over three semesters at each college — with 
one to two semesters of follow-up — to estimate whether students assigned to receive biweekly 
disbursements have better financial and academic outcomes than students assigned to receive 
their financial aid under the colleges’ standard lump sum policy. 23 A final report scheduled for 
2018 will include findings for over 2,500 additional students, with two to six semesters of 
follow-up. MDRC has also launched a third phase of the evaluation to examine implementation 
of the policy at two colleges in the West Hills Community College District in California’s rural 
Central Valley. At these schools, biweekly disbursements of financial aid refunds became 
policy for all students in the 2016-2017 academic year. This third phase will enable MDRC to 
study how the policy of biweekly aid disbursements works in a different setting and when 
implemented as a campus-wide policy, rather than applied only to a subset of students. 

MDRC researchers have collected multiple fonns of institutional and site visit data that 
describe the implementation of Aid Like A Paycheck at HCC and San Jacinto, and student-level 
financial aid, account, and academic transcript data are being collected for every student in the 
study sample. Between fall 2014 and spring 2016, MDRC conducted focus groups and inter¬ 
views with more than 50 students in the program and standard groups, more than 25 financial 
aid advisers or office staff members, and 11 administrators. The purpose of the focus groups 
and interviews was to better understand the context in which biweekly refunds were imple¬ 
mented, how these different stakeholders experienced the policy’s implementation, and how 
these factors provide insight into the treatment contrast and impact findings. Additionally, 
approximately 1,000 students (half in the program group and half in the standard group across 
both study sites) were invited to complete a survey once per month throughout their first 
semester in the Aid Like A Paycheck study. 


Description of the Study Colleges and Students 

The two colleges participating in the Aid Like A Paycheck randomized controlled trial are 
located in the metropolitan area of Houston. HCC is the fifth largest community college 
system in the country, comprising six colleges and more than 15 campuses spread across the 

22 The Aid Like A Paycheck analysis plan (2014) is posted at: https://osf.io/ext8h/. 

23 Financial aid data presented in this report are from students’ first semester in the study; academic data 
are from students’ first and second semesters. The first program semesters at HCC were spring 2015, fall 2015, 
and spring 2016. The first program semesters at San Jacinto were fall 2014, spring 2015, and fall 2015. 
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city, and serving about 90,000 students per academic year. 24 HCC offers more than 100 areas 
of academic study and over 50 certificate and skills training programs. 25 San Jacinto College 
has three campuses in Houston’s southeastern suburban region, serves about 40,000 students 
per academic year, and offers over 185 academic and career technical programs. 26 

Students in this study were selected through college records based on the criterion of 
having an expected financial aid refund amount of $800 or more after tuition and fees were 
paid, allowing for book expenditures. (This cutoff amount was selected so that the biweekly 
disbursements would not be very small.) Table 1 shows several demographic and financial aid 
characteristics of students at the time of randomization, before their first program semesters 
began. Students participating in the Aid Like A Paycheck study were 23 years old on average, 
with students at HCC somewhat older and less likely to be dependent on their parents than those 
at San Jacinto. At HCC, roughly one-third of students in the sample are Hispanic, and over half 
are black. At San Jacinto, two-thirds are Hispanic. At both colleges, over half the participating 
students are female. 

Students’ planned academic load varied between the two colleges during the 2014-2016 
academic years. At HCC, roughly the same proportions of students in the sample were enrolled 
full time, three-quarter time, or half time. At San Jacinto, students in the sample enrolled in 
more courses on average, with nearly two-thirds of students enrolled full time and about one- 
third enrolled three-quarter time. About one-third of students in the sample at HCC needed 
developmental courses, while over half did at San Jacinto. 

By design of the program and study, an overwhelming majority of students in both col¬ 
leges’ samples received Pell Grants. Far more students expected to take out loans in the HCC 
sample than the San Jacinto sample (about half compared with about one-sixth), causing the 
average expected refund at HCC to be substantially higher. This difference is reflective of the 
general trends regarding loans at the two colleges: 22 percent of full-time beginning undergrad¬ 
uate students take out loans at HCC, compared with only 6 percent at San Jacinto. 27 As a result 
of the colleges’ decisions regarding student enrollment in the study, all students in HCC’s 
sample were new to HCC and had never received financial aid at the college before, while the 
San Jacinto sample includes students who had attempted up to 14 credits at the college and may 
have received aid in prior semesters. 


24 Houston Community College (2016); Integrated Postsecondary Education Data System (IPEDS), U.S. 
Department of Education, Institute of Education Sciences, National Center for Education Statistics 
(https://nces.ed.gov/ipeds). 

25 Houston Community College (2017a, 2017b). 

26 San Jacinto College District (2016); IPEDS. 

27 IPEDS. 
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Table 1 


Characteristics of Aid Like A Paycheck Study Participants, Houston Community College 
and San Jacinto College, Fall 2014 to Spring 2016 Cohorts 


Characteristic 

Houston Community College 

San Jacinto College 

Age (years) 

24.7 

20.8 

Male (%) 

43.7 

40.1 

Race/ethnicity (%) 

Hispanic 

33.0 

63.6 

White 

8.1 

16.8 

Asian 

6.1 

4.0 

Black 

51.5 

14.7 

Other 

0.8 

0.0 

Planned academic load at baseline (%) 

Less than half time (1 to 5 credits) 

6.4 

0.0 

Half time (6 to 8 credits) 

30.4 

6.5 

Three-quarter time (9 to 11 credits) 

26.1 

32.4 

Full time (12 credits or more) 

37.1 

61.0 

In need of developmental education (%) 

34.7 

53.6 

Cost of attendance 3 ($) 

11,090 

9,897 

Expected Family Contribution greater than zero (%) 

24.0 

17.1 

Expected refund ($) 

3,285 

2,479 

Expected financial aid package includes 13 (%) 

Pell grant 

90.9 

100.0 

Pell only 

31.3 

55.2 

State aid 

24.9 

30.5 

Loans 

51.0 

17.1 

Other aid/scholarships 

1.4 

1.7 

Received aid in prior semesters (%) 

0.0 

17.5 

Prior credits attempted 3 (%) 

None 

100.0 

84.3 

1-14 

0.0 

15.7 

Dependent on parents (%) 

51.6 

71.6 

Sample size (total = 6,264) 

2,980 

3,284 


SOURCE: MDRC calculations using baseline data from Houston Community College and San Jacinto College. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

The first program semesters for the three student cohorts were fall 2014, spring 2015, and fall 2015 at San 
Jacinto College and spring 2015, fall 2015, and spring 2016 at Houston Community College. 

“Cost of attendance refers to the average cost of attending college full time for one academic year (fall through 
spring), including tuition and fees, books and supplies, room and board, transportation, and personal expenses, 
distributions do not sum to 100 percent because categories are not mutually exclusive. 
c At San Jacinto College, prior credits attempted includes courses from semesters before random assignment. 
Courses that students were enrolled in at the time of random assignment are not included. 
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Monthly surveys of students in the sample assigned to receive their aid as a standard 
lump sum disbursement may provide insights into the typical financial conditions of students at 
these colleges. 28 While the survey results provide the context of the student experience at HCC 
and San Jacinto, the response rates suggest that the survey findings — particularly the differ¬ 
ences between program and standard groups, and the moderate changes seen over time — 
should be interpreted with caution. 

As seen in Figure 3, survey findings from the first month of the semester indicate that 
around 40 percent of respondents assigned to receive lump sum aid at both colleges agreed (or 
strongly agreed) that their “finances cause significant stress.” The percentage of San Jacinto 
respondents who reported the same at the end of the semester increased somewhat, while the 
percentage of HCC respondents remained about 40 percent. Roughly one-quarter of respond¬ 
ents in the standard group at both colleges reported feeling worried “often” or “always” in the 
past week about paying bills, and about having money to spend on daily needs, both during the 
first month of the semester and during the fourth month of the semester. 29 

Survey results of students assigned to standard lump sum aid also show that many spent 
a considerable amount of time working while enrolled in classes. At HCC, about half the survey 
respondents in the standard group reported that they were currently working for pay when asked 
in the first month of the semester. At San Jacinto, this portion was about 40 percent. This 
suggests that the need to work while taking classes may have been greater at the larger urban 


28 About 1,000 students were randomly selected from the program and standard groups and invited to re¬ 
spond to a survey about their academic and financial attitudes, behaviors, and circumstances. The survey 
questions were developed primarily by MDRC, in partnership with the Center for Financial Services Innova¬ 
tion and the Center for Community College Student Engagement. The survey was administered once per 
month for four months during the students’ first semester in the study at HCC and San Jacinto. The average 
response rate was nearly 60 percent, but it differed somewhat across months, colleges, and research groups. 
The surveys administered in the first and fourth months generally had the highest response rates. Surveys in the 
second and third months had somewhat lower rates, and because the findings in those months do not indicate 
notable changes in trends, this report focuses on responses in the first and fourth months. Full survey responses 
for each month at both colleges, as well as a description of response rates and potential response bias, are 
available upon request to ALAP@mdrc.org. 

29 Findings from this study pose an interesting contrast to findings from a financial health survey by the 
Center for Financial Services Innovation in which one-quarter of low-income households surveyed reported 
feeling significant stress due to their finances (a smaller portion than the respondents in the Aid Like A 
Paycheck survey sample). However, over half of those households reported difficulties paying their bills on 
time, and one-quarter reported having too much debt (Gutman, Garon, Hogarth, and Schneider, 2015). In 
contrast, and despite their higher levels of reported stress, the majority of Aid Like A Paycheck survey 
respondents assigned to receive standard aid disbursements agreed or strongly agreed that they “usually pay 
[their] bills on time,” that they “know how to manage [their] finances well,” and that they budget weekly or 
daily. Moreover, very few students reported agreeing or strongly agreeing that they had “too much debt right 
now.” 
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Figure 3 


Month 1 and Month 4 Survey Responses from Standard Group Students 
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San Jacinto College 


Financial health 
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Worried about money 



Work behavior 



48 



31 



Currently works 
for pay 

Worked more than 20 
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Currently looking for 
work 



Percentage 


Percentage 


SOURCE: MDRC calculations using responses from the Aid Like A Paycheck student surveys at Houston 
Community College (HCC) and San Jacinto College. 

NOTES: At HCC, the Aid Like A Paycheck student survey was sent monthly to 236 students in the standard group. 
In Month 1, 142 students responded; in Month 4, 139 students responded. At San Jacinto, the Aid Like A Paycheck 
student survey was sent monthly to 235 students in the standard group. In Month 1, 147 students responded; in 
Month 4, 141 students responded. 


college, which had more students who were independent of their parents, than at the suburban 
college with younger students. At both colleges, the portion of respondents who reported 
currently working for pay increased somewhat when students were asked again during the last 
month of the semester, although this observed moderate increase over time may not be repre¬ 
sentative of the full sample. 

During small student focus groups at both schools early in the semester, students who 
received their aid in the standard lump sum disbursement often stated that their normal life costs 
— such as rent, child care, and groceries — prompted them to continue working while taking 
courses, because financial aid alone would not have been enough to cover all their expenses. A 
few students expressed fear of losing their jobs if they were to attempt to work fewer hours 
while attending school. 
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When standard group students were surveyed in the first and fourth months of the se¬ 
mester, about 40 percent of HCC respondents and almost one-quarter of San Jacinto respond¬ 
ents reported working more than 20 hours per week. This is important given research suggest¬ 
ing that students who work more than 20 hours a week during their first college year suffer 
academically as a result. 30 While notable percentages of respondents from both colleges 
reported working while taking courses, the amount of time they reported working is lower than 
national findings from the Community College Survey of Student Engagement (CCSSE), in 
which half the students reported working more than 20 hours per week. 31 

When it came to their academic behaviors (not shown in Figure 3), most survey re¬ 
spondents in the standard aid group reported that they spent at least 6 to 10 hours a week 
studying and attended all their classes (although the proportion declines somewhat between the 
first and last month of the semester), which is consistent with CCSSE national survey find¬ 
ings. 32 In small focus groups, some working students mentioned their struggles to find enough 
time to study because of their work schedules, and others reported that their work hours took 
time and energy, preventing them from completing their course assignments to the best of their 
abilities. 

The survey and focus group responses of students assigned to receive standard lump 
sum financial aid disbursements do not fully support some of the assumptions made in Aid Like 
A Paycheck’s theory of change. For instance, the theory of change suggests that receiving 
financial aid in a lump sum at the beginning of the semester could result in students’ experienc¬ 
ing large increases in financial stress or worry by the end of the semester. However, the survey 
findings indicate that for respondents assigned to the standard group at HCC, there was no 
increase from the beginning to the end of the semester in the percentage who reported that their 
finances caused them significant stress, and at San Jacinto, the change in survey responses over 
time does not appear large enough to attribute with confidence to students feeling hardship due 
to spending their aid before the end of the term. 

The prediction that students receiving their aid in a lump sum would be more likely to 
report working for pay later in the semester than at the start was not strongly substantiated by 
either the survey or focus group findings. Many survey respondents assigned to lump sum aid 
from both colleges reported working at various points in the semester, but there was only a 
moderate increase in the percentage of respondents who reported working near the end of the 


30 Pike, Kuh, and Massa-McKinley (2008). 

31 Center for Community College Student Engagement (2015). 
32 Center for Community College Student Engagement (2015). 
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semester. In focus groups, students reported that they often made choices to try to keep their 
work hours consistent throughout the semester. 

Finally, the prediction that receiving financial aid in the standard lump sum at the be¬ 
ginning of the semester could lead students to mismanage their finances or to regress in 
academic behaviors did not appear to be reflected in the survey responses of students assigned 
to the standard group. Most respondents reported being able to manage their money well and 
to prepare for and attend classes regularly, although anecdotal evidence from focus group 
participants suggested that some students are unable to make their aid refunds last until the 
end of the semester. 


Implementation of Aid Like A Paycheck 

Mechanical Aspects of Aid Disbursements 

Both colleges revised their financial aid software systems to automate biweekly dis¬ 
bursements for students assigned to the Aid Like A Paycheck program. As noted above, the 
colleges selected students with an expected refund of over $800 for enrollment in the study, and 
MDRC randomly assigned students to receive refunds according to each college’s rules and 
procedures for biweekly or standard lump sum disbursements. 

• Students assigned to the program group were expected to receive seven 
roughly equal disbursements throughout the term. In practice, most students 
in the program group received six to eight refunds, averaging between $300 
and $500 per refund. 

• Students assigned to the standard group followed their college’s standard 
policies and procedures, which generally entailed a lump sum disbursement 
of grant aid about three or four weeks into each tenn, sometimes followed by 
a lump sum disbursement of loans soon after. In practice, most students in 
the standard group received one or two refunds, averaging between $1,500 
and $2,000 per refund. 

• Variations from the intended refund schedules were typically the result of 
financial aid rules and regulations or changes in students’ circumstances, 
rather than mistakes in the processing or refunding of aid. 

To calculate and generate biweekly disbursements, each college created policies based 
on federal financial aid rules and regulations, its own internal financial aid policies, and its 
vision for implementing Aid Like A Paycheck. At both colleges, creating and implementing 
these policies was challenging and required adjustments — often completed manually — based 
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on student enrollment changes, late-start classes, loan changes, and other factors. While the 
overall programs looked similar, there were notable differences in the colleges’ policies. For 
example, at San Jacinto, students in the program typically received their first biweekly refund 
one week before the standard lump sum refunds at the college, while at HCC, the first biweekly 
refund was typically at the same time as the standard refund. At San Jacinto, students in “part- 
of-tenn” (less than 16-week) courses received aid based on their current active course load; 
students in both program and standard groups may have received additional aid later as they 
began late-start courses. At HCC, biweekly aid amounts were calculated based on the number 
of credits a student had enrolled in, including part-of-tenn courses that began later in the 
semester, thus providing more steady aid throughout the semester for students who registered 
early for late-start courses. The implications of these decisions on student outcomes are briefly 
considered in the conclusions of this report. 

The majority of students assigned to receive biweekly disbursements at both colleges 
did receive evenly timed refunds throughout the semester, as intended. 33 Financial aid records of 
students assigned to receive biweekly aid indicate that about 60 percent of HCC program group 
students and nearly 75 percent of San Jacinto program group students received between six and 
eight refunds at regular intervals during the semester, as shown in Tables 2 and 3. Students’ 
refunds were often in roughly equal amounts, though for at least half the students, one or more 
of the refunds were substantially higher or lower than the others. These variations were trig¬ 
gered by factors such as students adding or dropping classes, which changed the amount of 
tuition due and eligible aid; taking out loans after the start of the term; or using aid not yet 
disbursed for large purchases at the bookstore. Fewer than 10 percent of students assigned to 
receive biweekly disbursement ended up receiving only a single disbursement; of these stu¬ 
dents, the overwhelming majority received only one refund because their amount turned out to 
be below the $800 threshold. 

Nearly all students assigned to receive standard aid as a lump sum received between 
one and three refunds per semester, as shown in Tables 2 and 3. Some students in the standard 
group received more than one disbursement because of federal loans, which have their own 
timing, and because of changes in enrollment. Fewer than 3 percent of students in the standard 
group received more than three refunds in the semester at either college. 


33 At San Jacinto, students were permitted to request hardship exemptions from the biweekly disbursement 
policy; if granted, the college provided the student with the full remaining refund amount in a lump sum for the 
semester. During the three semesters covered in this report, there were 14 requests submitted and four 
exemptions granted by San Jacinto. 
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Table 2 


Refund Frequency and Amount Among Enrolled Study Participants, 
First Program Semester, Houston Community College 


Number of Refunds 

Percentage 

Average Refund Amount ($) a 

Program 

Group 

Standard 

Group 

Program 

Group 

Standard 

Group 

0 

4.6 

3.4 

0 

0 

1 

8.3 

49.4 

554 

1,677 

2 to 3 

9.7 

44.8 

540 

1,688 

4 to 5 

15.3 

2.3 

584 

1,265 

6 to 8 

61.6 

0.1 

463 

— 

9 or more 

0.6 

0.0 

— 

— 

Sample size (total = 2,887) 

1,435 

1,452 




SOURCE: MDRC calculations using refund data from Houston Community College. 


NOTES: Rounding may cause slight discrepancies in sums and differences. 

The first program semesters at Houston Community College were spring 2015, fall 2015, and spring 
2016. 

a Amounts not shown for frequencies smaller than 1 percent. 


Table 3 


Refund Frequency and Amount Among Enrolled Study Participants, 
First Program Semester, San Jacinto College 


Number of Refunds 

Percentage 

Average Refund Amount ($) d 

Program 

Group 

Standard 

Group 

Program 

Group 

Standard 

Group 

0 

1.9 

1.1 

0 

0 

1 

4.7 

65.3 

502 

1,775 

2 to 3 

5.4 

32.4 

318 

1,158 

4 to 5 

11.5 

0.9 

321 

— 

6 to 8 

73.5 

0.2 

305 

— 

9 or more 

2.9 

0.1 

438 

— 

Sample size (total = 3,196) 

1,603 

1,593 




SOURCE: MDRC calculations using refund data from San Jacinto College. 


NOTES: Rounding may cause slight discrepancies in sums and differences. 

The first program semesters at San Jacinto College were fall 2014, spring 2015, and fall 2015. 
a Amounts not shown for frequencies smaller than 1 percent. 
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Messaging About Aid Like A Paycheck 

• College communications about biweekly disbursements — and about finan¬ 
cial aid in general — were often unclear to students, although colleges made 
efforts to improve them. 

Both colleges initially incorporated infonnation about the new Aid Like A Paycheck 
disbursement policy into their financial aid websites and into printed and emailed messages 
about financial aid. Once students had been assigned to receive biweekly or standard lump sum 
disbursements, the colleges sent email notifications to those students infonning them of their 
assigned refund method. Both colleges held training for financial aid staff members to introduce 
Aid Like A Paycheck and to assist staff members in explaining the biweekly disbursement 
policy to students. 

Several financial aid staff members from both colleges, however, indicated that many 
students do not read financial aid infonnation closely and do not understand many aspects of 
financial aid. Indeed, many students assigned to receive biweekly disbursements did not appear 
to understand the new policy and were confused about why they were not receiving all their aid 
in a lump sum. In response to early confusion among students assigned to biweekly disburse¬ 
ments — as well as among the financial aid staff about how to respond to the students’ ques¬ 
tions — financial aid administrators made considerable efforts to make the messaging about the 
policy change clearer and more consistent. The main communication challenge was to provide 
real-time infonnation to students in the program group about the expected timing and amount of 
their refunds, particularly when enrollment changes or bookstore purchases would cause their 
future refund amounts to change. 

At HCC — the larger, less centralized college system — consistency in delivering up¬ 
dated financial aid infomiation across its several colleges proved more difficult than the task at 
San Jacinto. In interviews, many financial aid front desk and call center staff members reported 
having difficulties understanding how the district office adjusted students’ refunds and being 
unsure what to tell students regarding the timing and amount of their refunds. Some financial 
aid staff members also struggled to understand how students’ disbursements should be calculat¬ 
ed when loan amounts were factored in. 

Staff members at San Jacinto faced similar difficulties in communicating with students 
about Aid Like A Paycheck, but to a lesser degree. While financial aid advisers and coordina¬ 
tors at San Jacinto reported being well infonned about how to discuss biweekly disbursements 
with students, some call center and business office staff members stated that they did not 
understand the details of the policy change and thus had little specific infonnation to pass on to 
students. 
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In order to address the communication gaps regarding the policy change for program 
group students, both colleges bolstered training of all financial aid staff members on how to 
address student questions. For example, after the first semester of implementing biweekly 
disbursements, financial aid administrators at each college held staff training on how to identify 
students in the program, how to calculate aid in even disbursements, and when students should 
expect to receive their refunds. 

The theory of change for Aid Like A Paycheck hypothesizes that financial aid offices 
would change their communications for students receiving biweekly disbursements in a way 
that could help students perceive an incentive to continue in classes, help them budget their aid 
and other income to last throughout the semester, and allow them to balance their time working 
and studying. However, given the challenge of clearly communicating about financial aid in 
general — which was magnified in the effort to communicate about biweekly disbursements — 
the implementation of Aid Like A Paycheck’s messaging component appears to have been 
weaker than the ideal model. 


Students’ Experiences in the Aid Like A Paycheck Study 

This section of the report examines how students experienced and reacted to the way in which 
their aid was disbursed, as well as to the messages they received about it. The findings are based 
primarily on survey responses during students’ first semester in the study. Overall, the survey 
provides rich infonnation; however, different response rates between students in the program 
and standard groups in the first month at HCC suggest that the results in that month should be 
interpreted with caution. Focus groups of students were also conducted near the beginning and 
end of each semester; the findings were combined with the survey results to describe students’ 
experiences. 


• In the first month of the semester, survey respondents who were assigned to 
receive biweekly disbursements appeared more likely to report receiving aid 
payments that were “less than expected” than students assigned to receive the 
standard lump sum aid. 

• In the first month of the semester, respondents assigned to receive biweekly 
refunds appeared more likely to report feelings associated with negative 
financial health. For example, they were more likely to agree that their “fi¬ 
nances cause significant stress” than those assigned to receive a lump sum 
refund. At the end of the semester, these differences were generally no 
longer evident. 
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• About half the survey respondents reported working while enrolled in classes 

both at the beginning and toward the end of the semester, regardless of how 
they received their aid. 

Expectations for Aid Amounts and Timing 

Although the majority of students in the study were new students, many probably had 
expectations of receiving their financial aid in the standard lump sum disbursement based on 
what they had heard from their peers, friends, or family members who had received financial 
aid in the past. When surveyed in the first month of their semester, many students at both 
schools who were assigned to receive biweekly disbursements reported that their first or second 
refund amounts differed from what they had expected. As indicated in the top panel of Table 4, 
over half the survey respondents at HCC assigned to receive biweekly disbursements reported 
that their early aid refunds were “less than expected,” compared with only one-third of students 
receiving a standard disbursement, a statistically significant difference. When asked about the 
timing of their disbursements, about one-third of HCC students assigned to biweekly disburse¬ 
ments reported that their early semester refunds “came after [they] had expected,” compared 
with only about one-fifth of students who received a lump sum. 

A statistically significant difference in expectations of aid amounts was also evident at 
San Jacinto, where about 40 percent of survey respondents assigned to biweekly disbursements 
reported in the first month that they had received “less than expected” amounts, compared with 
only about 20 percent of students receiving standard lump sum aid. At San Jacinto, where the 
first biweekly disbursement was typically one week before the standard lump sum, about 25 
percent of students in each group reported receiving their refund later than expected. 

Most students in the program group who participated in small focus groups early in the 
semester stated that they were generally surprised to learn they were going to receive biweekly 
refunds rather than a lump sum, and many felt frustrated by receiving refunds less than or later 
than they expected. These incorrect assumptions about their aid existed despite emails from the 
financial aid offices describing how they would receive their aid. Some students in the program 
group noted that they were particularly frustrated when they learned of other students who had 
received lump sum refunds. 

By the last month of the semester, however, a large majority of students assigned to bi¬ 
weekly disbursements at both colleges reported that their most recent refund was neither smaller 
nor later than what they expected to receive. (Because most students in the standard group 
typically received refunds only early in the semester, these questions were not applicable and 
their survey responses are not included.) 
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Table 4 


Financial Health, Work Behavior, Academic Behavior, and Financial Aid Expectations, 
Houston Community College and San Jacinto College 





Month 1 



Month 4 




Program Standard 


Program Standard 


Survey Item (%) 


Group 

Group 

P-Value 

Group 

Group 

P-Value 

Financial aid expectations 3 








Amount of most recent 

HCC 

58.8 

31.8 *** 

0.001 

9.2 

— 


refund payment was less 
than expected 

San Jacinto 

39.3 

20 7 *** 

0.003 

6.5 

— 


Timing of most recent 

HCC 

35.0 

17.7 ** 

0.018 

13.3 

— 


refund payment came 
after expected 

San Jacinto 

24.7 

25.6 

0.881 

19.6 

— 


Financial health 








Strongly agree or agree 

HCC 

55.6 

42.1 ** 

0.018 

47.4 

39.7 

0.185 

that finances cause 
significant stress 

San Jacinto 

48.1 

37.7 * 

0.080 

43.5 

47.9 

0.465 

Often or always worried 

HCC 

32.5 

25.9 

0.205 

37.4 

28.2 * 

0.095 

about paying bills on time 

San Jacinto 

27.9 

21.8 

0.239 

29.7 

30.5 

0.887 

Often or always worried 

HCC 

35.5 

22.9 ** 

0.015 

30.3 

27.4 

0.587 

about money to pay for 
daily needs 

San Jacinto 

28.5 

26.0 

0.651 

24.6 

28.4 

0.482 

Work behavior 








Currently works for pay 

HCC 

53.0 

50.7 

0.694 

54.5 

59.9 

0.356 

during the semester 

San Jacinto 

50.0 

42.2 

0.194 

55.8 

48.2 

0.207 

Spent more than 20 hours 

HCC 

36.3 

37.9 

0.780 

34.6 

40.4 

0.306 

working for pay in the past 

7 days 

San Jacinto 

25.4 

22.5 

0.569 

31.2 

24.8 

0.240 

Currently not working for 

HCC 

32.1 

31.4 

0.894 

31.4 

26.3 

0.336 

pay, but looking for work 

San Jacinto 

25.4 

34.0 

0.119 

24.6 

32.6 

0.141 

(continued) 


Students’ Financial Health 

Like their peers assigned to receive aid as a lump sum, notable percentages of respond¬ 
ents assigned to biweekly disbursements reported financial stress. Neither the survey nor focus 
group findings offer evidence for the prediction in the theory of change that program group 
students would experience less financial stress or worry as a result of receiving aid in biweekly 
disbursements. 
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Table 4 (continued) 


Survey Item (%) 



Month 1 



Month 4 


Program Standard 
Group Group 

P-Value 

Program Standard 
Group Group 

P-Value 

Academic behavior 








Spent more than 10 hours 

HCC 

29.0 

33.1 

0.440 

35.3 

30.7 

0.406 

studying in the past 7 days 

San Jacinto 

35.6 

28.6 

0.210 

40.6 

37.9 

0.643 

Attended all classes in the 

HCC 

74.6 

gj g *** 

0.003 

73.3 

73.7 

0.927 

past 7 days 

San Jacinto 

90.9 

87.1 

0.311 

85.5 

76.6 * 

0.058 

Sample size 








Respondent (HCC) 


171 

142 


157 

139 


Total (HCC) 


263 

236 


263 

236 


Respondent (San Jacinto) 


134 

147 


138 

141 


Total (San Jacinto) 


222 

235 


222 

235 



SOURCE: MDRC calculations using responses from the Aid Like A Paycheck surveys at Houston Community 
College (HCC) and San Jacinto College. 

NOTES: In the first survey month at HCC (shown in italics above), the combination of an overall response rate 
of around 60 percent and a 5 percent differential in response rates between the program and standard groups 
may have resulted in biased estimates. Consequently, these results could receive a rating no higher than "meets 
evidence standard with reservations" by the What Works Clearinghouse standards. During Months 2 and 3 (not 
shown above), similar bias may have resulted from the combination of response rates and differentials between 
research groups at both colleges. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as *** = 1 percent; ** = 5 percent; * = 10 percent. 

a Due to standard financial aid disbursements occurring in the beginning of the term, questions regarding 
financial aid expectations in Month 4 of the term were not applicable to the majority of standard students. Thus, 
the few responses are not presented here. 


Table 4 shows that in the first month of the semester at HCC, over half the survey re¬ 
spondents assigned to biweekly disbursements “agreed” or “strongly agreed” that their “financ¬ 
es cause significant stress,” significantly more than the 42 percent of respondents assigned to 
lump sum disbursements. About one-third of HCC respondents assigned to biweekly disburse¬ 
ments also indicated that they worried “often” or “always” about paying their bills or having 
enough money for their daily needs in the past week at the beginning of the semester. This 
portion is higher than for students in the standard group. 

At San Jacinto, nearly half the survey respondents assigned to biweekly disbursements 
agreed that their “finances cause significant stress” at the start of the term, significantly more 
than respondents in the standard group. Just over one-quarter of San Jacinto respondents 
assigned to biweekly disbursements reported worrying “often” or “always” about paying bills, 
which is more than for respondents in the standard group, although the difference is not statisti- 
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cally significant. Just over one-quarter of respondents in both groups reported worrying about 
having money for daily needs at the beginning of the semester. 

These topics of stress and worry related to finances were discussed in small student fo¬ 
cus groups at both colleges at the beginning of the semester. Students receiving biweekly 
disbursements at both schools mentioned wanting more of their aid up front to purchase more 
expensive items — such as a laptop or other school supplies — or to cover other living and 
transportation costs at the start of the semester. A few focus group students who were enrolled 
in career and technical programs — such as welding, automotive, and nursing programs — 
were required to purchase specific equipment early in the semester that was not available for 
purchase in the campus bookstore; their initial aid allotment was not sufficient to cover these 
supplies, and they had to find other ways to cover the costs. 

By the last month of the semester, when surveyed again, respondents assigned to the 
program group at HCC were still more likely than those assigned to receive standard financial 
aid to report feeling financial stress or worry, although the differences were smaller and not 
statistically significant for two of the three measures. At San Jacinto, respondents in the pro¬ 
gram and standard groups reported roughly equal amounts of financial stress and worry at the 
end of the semester, suggesting that the program neither increased nor reduced students’ 
financial health by these measures. 

Students’ Work Behavior 

Much like their peers who were assigned to standard aid, about half the respondents as¬ 
signed to biweekly disbursements at both colleges reported working for pay while taking 
classes, and many reported working over 20 hours during the previous week (Table 4). At both 
colleges, the portion of respondents in the program and standard groups who reported working 
in the past week increased somewhat from the start of the semester to the end of the semester. 
However, survey response rates and sample sizes are not sufficient to determine whether these 
small changes over time, or the differences between the program and standard groups, are due 
to the intervention or to chance. 

Student focus groups conducted at both colleges revealed that —just like students re¬ 
ceiving lump sum aid — many of the students receiving biweekly refunds felt pressed to work 
while attending college, in contrast to the theory of change. Some working students with 
biweekly disbursements stated that they could not reduce their work hours because their jobs 
required a fixed number of hours, or because they feared losing their jobs if they tried to reduce 
their hours. Some students receiving biweekly disbursements mentioned in focus groups that 
they decided to work more hours or seek out second jobs after finding out they would not 
receive their full refund early in the semester. Many students receiving biweekly or standard 
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lump sum disbursements, at both colleges, reported that the need to earn money hindered their 
ability to find time to study. 

Students’ Academic Behavior 

At both colleges, between about 30 percent and 40 percent of respondents in the pro¬ 
gram and standard groups reported spending more than 10 hours studying in the prior week. 
There is no indication of significant differences between the groups, or of significant changes 
over time. Some statistically significant differences appear in students’ reporting of attendance; 
most notably, when asked near the start of the semester if they had attended all their classes in 
the prior week, HCC students assigned to receive biweekly disbursements were less likely to 
respond “yes” than those assigned to receive a standard lump sum. However, as discussed 
below in the “Financial and Academic Impacts” section, academic records for the full sample of 
students at HCC do not indicate a difference in students’ enrollment or success in earning 
credits. 


General Perceptions of Aid Like A Paycheck 

While the majority of focus group students understood the reasoning behind their col¬ 
leges’ decision to roll out aid disbursements throughout the semester, it was still a source of 
frustration. During small focus groups early in the semester, most students mentioned that 
although biweekly refunds seemed “like a good idea” for others who might otherwise spend all 
of their aid at once, they themselves “already know how to manage money” and did not need 
the program to spread their aid throughout the semester. Some students who were receiving 
biweekly disbursements and were not working at the time agreed that they would have preferred 
to receive their aid in a lump sum rather than on a biweekly schedule. Focus group participants 
who were older and those who identified as parents tended to be most vocal about not wanting 
their aid divided incrementally throughout the semester. 

Later in the semester, however, the majority of students who participated in focus 
groups and received biweekly disbursements reported that they liked receiving their aid incre¬ 
mentally and preferred to continue receiving their aid through biweekly disbursements. These 
late-semester focus group participants at both colleges reported feeling that biweekly disburse¬ 
ments “rewarded” them as they progressed through the semester and provided them with more 
“self-control” in spending their money and budgeting. These students also expressed the belief 
that if they had received a lump sum, they would have spent it quickly, leaving them with little 
or no refund money later in the semester. 

In comparison, most students who received standard lump sums reported in focus 
groups at the beginning and end of the semester that they preferred receiving a lump sum 
because they were able to have more control over their money, their spending habits, and their 
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ability to save. Some mentioned that they believed getting their refund in biweekly disburse¬ 
ments would have been “stressful” or “inconvenient” due to the need monitor their payment 
amounts every few weeks. Like the program group students at the beginning of the semester 
who did not like biweekly disbursements, many students who received a lump sum agreed that 
there could be value in biweekly disbursements but that such a disbursement schedule would be 
especially beneficial for students who were less diligent with money management than they 
believed themselves to be. 

It is interesting to note that working students who participated in the focus groups gen¬ 
erally did not express strong opinions about how they received their aid, reporting that they 
were relying on their employment income as their main financial source. This was the case for 
working students whether they received their aid biweekly or in the standard lump sum. 


Financial and Academic Impacts 

This section presents the estimated impacts of the Aid Like A Paycheck intervention on 
students who joined the study in the first three program semesters at San Jacinto and HCC, with 
one to two semesters of follow-up. The main findings are as follows: 

• Students assigned to receive biweekly refunds and those assigned to receive 
their aid in the standard way received the same total amount of financial aid, 
yet biweekly disbursements reduced the use of federal loans (particularly at 
HCC) and students’ debt to the college (at San Jacinto). 

• On average, there is little consistent evidence of biweekly disbursements im¬ 
proving students’ key academic outcomes. However, at San Jacinto, students 
in the biweekly disbursement group experienced a 6 percentage point in¬ 
crease in enrollment in the second semester of school. 

Financial Aid Outcomes 

Across colleges, during their first program semester, students in the program group re¬ 
ceived the same amount of total financial aid on average — about $3,300 — as those in the 
standard group, as shown in the first row of Table 5. This row, with a breakdown by aid types 
underneath, represents the financial aid disbursements, or the amounts of aid applied to a 
student’s account before subtracting tuition, fees, books, and other college costs. While total aid 
did not differ significantly, students in the program group received an average of $82 less in 
loan disbursements during the first semester; this impact is consistent with the goal of reducing 
student debt as described in the theory of change. 
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Table 5 


First-Semester Financial Aid Outcomes, Houston Community College and 
San Jacinto College, Fall 2014 to Spring 2016 Cohorts 


Outcome 

Program 

Group 

Standard 

Group 

Difference 

P-Value 

Financial aid received ($) 

3,286 

3,331 

-45 

0.329 

Federal grants 

2,110 

2,087 

23 

0.339 

State grants 

355 

352 

4 

0.787 

Other grants 

39 

29 

10 

0.102 

Loans 

782 

864 

_82** 

0.041 

Financial aid refunded net tuition and fees ($) 

2,143 

2,252 

-109*** 

0.004 

Students who owe debt to the college (%) 

8.0 

8.5 

-0.5 

0.443 

Outstanding debt to college including R2T4 ($) 

61 

85 

-24** 

0.014 

Outstanding debt to college, including R2T4, 





among those who owe 

783 

997 



Students with R2T4 penalties (%) 

3.3 

4.1 

-0.9* 

0.067 

R2T4 obligation ($) 

27 

27 

0 

0.962 

Among those penalized, R2T4 obligation 

822 

644 



Official or unofficial withdrawal from college (%) 

6.9 

8.1 

-1.3* 

0.055 

Sample size (total = 6,264) 

3,124 

3,140 




SOURCE: MDRC calculations using financial aid data from Houston Community College and San Jacinto 
College. 

NOTES: R2T4 = Return to Title IV. 

Rounding may cause slight discrepancies in sums and differences. 

Estimates are adjusted by cohort. 

The first program semesters for the three student cohorts were fall 2014, spring 2015, and fall 2015 at San 
Jacinto College and spring 2015, fall 2015, and spring 2016 at Houston Community College. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 


The college-specific analyses of financial aid received follow a pattern generally similar 
to the pooled results, with some exceptions. As Table 6 shows, the impact on loan disburse¬ 
ments is statistically significant only at HCC, where students in the program group borrowed 
about $130 less on average than those in the standard group in their first semester. This reduc¬ 
tion in borrowing is primarily a result of fewer students taking out loans (about 40 percent of 
students in the program group, compared with 44 percent in the standard group) at HCC (not 
shown). Of those who borrowed, their average loan amount was about $3,460 in each research 
group. At San Jacinto, the portion of students who take out loans is generally lower than at 
HCC, and there are no statistically significant differences between students in the program and 
standard groups on these measures. 
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Table 6 


First-Semester Financial Aid Outcomes by College, Fall 2014 to Spring 2016 Cohorts 



Houston Community College 



San Jacinto College 




Program 

Standard 



Program 

Standard 



Subgroup 

Outcome 

Group 

Group 

Difference 

P-Value 

Group 

Group 

Difference 

P-Value 

Difference 

Financial aid received ($) 

3,556 

3,662 

-107 

0.165 

3,042 

3,030 

12 

0.818 


Federal grants 

1,846 

1,824 

22 

0.532 

2,350 

2,327 

23 

0.445 


State grants 

306 

306 

1 

0.965 

399 

393 

6 

0.771 


Other grants 

11 

12 

-1 

0.909 

64 

44 

20 * 

0.075 

f 

Loans 

1,392 

1,521 

-129* 

0.060 

229 

266 

-37 

0.272 


Financial aid refunded net 
tuition and fees ($) 

2,441 

2,580 

-139** 

0.031 

1,873 

1,955 

-82* 

0.054 


Students who owe debt 
to the college (%) 

7.4 

6.1 

1.3 

0.150 

8.5 

10.7 

- 2 . 2 ** 

0.031 

ft 

Outstanding debt to 
college including R2T4 ($) 

35 

32 

3 

0.689 

85 

132 


0.006 

tff 

Outstanding debt to 
college, including R2T4, 
among those who owe 

509 

542 



1,014 

1,221 




Students with R2T4 
penalties (%) 

1.9 

3.1 

-1.3** 

0.028 

4.5 

5.0 

-0.5 

0.477 


R2T4 obligation ($) 

10 

17 

. 7 * 

0.071 

42 

36 

7 

0.462 


Among those penalized, 

R2T4 obligation 

526 

532 



936 

706 




Official or unofficial 
withdrawal from college (%) 

3.5 

5.7 

-2 2 *** 

0.004 

9.9 

10.3 

-0.4 

0.670 


Sample size (total = 6,264) 

1,485 

1,495 



1,639 

1,645 





(continued) 



Table 6 (continued) 

SOURCE: MDRC calculations using financial aid data from Houston Community College and San Jacinto College. 

NOTES: R2T4 = Return to Title IV. 

Rounding may cause slight discrepancies in sums and differences. 

Estimates are adjusted by cohort. 

The first program semesters for the three student cohorts were fall 2014, spring 2015, and fall 2015 at San Jacinto 
College and spring 2015, fall 2015, and spring 2016 at Houston Community College. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

A two-tailed t-test was applied to differences in impacts between subgroups. Statistical significance levels are 
indicated as: fff = 1 percent; ff = 5 percent; f = 10 percent. 


The loan impact observed at HCC may be partially the result of refund rules for stu¬ 
dents assigned to receive biweekly disbursements: Loans were disbursed only after all Pell 
Grants had been disbursed. For example, if a student’s financial aid refund is made up of one- 
third Pell and two-thirds loan funds, the first two refunds would be Pell funds; the third refund 
would be a combination of Pell and the loan; and the remaining refunds would be loan funds. 
This was proposed to reduce the likelihood of borrowing for students who initially applied to 
take out a loan but withdrew during the semester. 

Although the average amount of financial aid received was about the same for students 
in both research groups at each college, the amount of aid refunded to students — that is, the 
amount the student received directly as a bank deposit or refund check after tuition, fees, and 
other expenses — was about $100 less for students assigned to receive biweekly refunds 
compared with those assigned to receive the standard lump sum. This impact is apparent at both 
colleges. The most likely explanation is that charges that arise during the semester, including 
books, parking tickets, library fees, or additional tuition, can be subtracted by the college from 
the undistributed portion of refunds for students in the program group, lowering the total. 
Students in the standard group who had already received their full refund in the lump sum 
would have to pay any further charges out of pocket or add to their student debt. 

Measures of student debt, shown in the middle rows of Tables 5 and 6, indicate that 
being assigned to receive biweekly disbursements did not significantly affect students’ 
likelihood of owing debt to the college at the end of their first semester but did reduce the 
average amount of such debt by $24 in the pooled sample. This impact is in line with the 
hypotheses put forth in the theory of change. As indicated in Table 6, there was no impact on 
debt to the college at HCC, but at San Jacinto, the program reduced both the portion of 
students who owed (by about 2 percentage points), and the average amount owed (by about 
$50). When accounting only for those students who had debt to the college at San Jacinto, 
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those in the program group owed about $200 less than their standard group counterparts, 
whose average debt to the college was about $1,200. 

Why would biweekly disbursements cause the differences observed in debt to the col¬ 
lege? One factor is that when students reduce their course load from full time to part time after 
receiving their full refund, colleges may charge them for the previously distributed aid in excess 
of their earned aid amount; students who receive their aid in a lump sum may be more likely to 
be subject to these charges. When a student is unable to pay, such charges show up as debt to 
the college. 

Student debt may also arise from Returns to Title IV (R2T4): When a student with¬ 
draws before completing 60 percent of the semester, the college may have to return financial aid 
to the federal government and may pass this penalty on to students. Although biweekly dis¬ 
bursements were hypothesized to reduce R2T4, there is no significant impact on the average 
amount of R2T4 in the pooled sample. However, students in the program group were identified 
as having withdrawn from the college at a lower rate (6.9 percent as opposed to 8.1 percent in 
the standard group; Table 5) in the first program semester, leading to somewhat lower rates of 
R2T4 penalties for these students. This impact was driven by HCC, as can be seen in Table 6. 
At HCC, the amount of R2T4 obligation (the amount the college pays back to the government) 
is significantly lower on average for students in the program group. This obligation concerns 
only the small number of students who withdraw during the semester. The lack of an observed 
impact at San Jacinto may be caused in part by the timing of disbursements there, which differs 
from HCC. As mentioned above, San Jacinto students received their first biweekly refund about 
a week before the standard lump sum refund. This means that some students in the standard 
group who withdrew or stopped attending courses in that timeframe would not receive the lump 
sum, but if they were assigned to receive biweekly disbursements in the program group, they 
would have already received the first incremental disbursement. This could lead to an increase 
in R2T4 for some program group students, thus partially offsetting any decrease in R2T4 from 
program group students who withdrew later in the semester. 

Academic Outcomes 

As shown in the bottom panel of Table 7, there is no pooled impact on persistence, de¬ 
fined as enrollment in the second program semester; this finding is counter to the theory of 
change. However, as shown in Table 8, at San Jacinto persistence was 6 percentage points 
higher for students assigned to receive biweekly disbursements. This is a large and statistically 
significant impact on a key outcome in the theory of change. Increasing persistence is a primary 
goal for many college reform efforts, because it can be a key step to improving credit accumula¬ 
tion and completion rates over time. Additionally, the increase in persistence may lead to more 
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Table 7 


Academic Outcomes, Houston Community College and San Jacinto College, 

Fall 2014 to Spring 2016 Cohorts 


Outcome 

Program 

Group 

Standard 

Group 

Difference 

P-Value 

First program semester 

Enrolled (%) 

97.2 

97.0 

0.3 

0.518 

Credits attempted 

10.8 

10.7 

0.1 

0.246 

Credits earned 

7.4 

7.4 

0.0 

0.743 

Sample size (total = 6,264) 

3,124 

3,140 



Second program semester 

Enrolled (%) 

69.5 

67.6 

1.9 

0.161 

Credits attempted 

7.6 

7.3 

0.3* 

0.066 

Credits earned 

5.3 

5.2 

0.1 

0.694 

Sample size (total = 4,220) 

2,105 

2,115 




SOURCE: MDRC calculations using transcript data from Houston Community College and San Jacinto College. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

Estimates are adjusted by cohort. 

The first program semesters for the three student cohorts were fall 2014, spring 2015, and fall 2015 at San 
Jacinto College and spring 2015, fall 2015, and spring 2016 at Houston Community College. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 


students in the program group receiving aid in future terms, when compared with students in the 
standard group who are less likely to be enrolled; this will be explored in the final Aid Like A 
Paycheck report. 

At San Jacinto, there is also a small impact on enrollment in the first program semester. 
While it is possible that an aspect of the messaging to students in the biweekly disbursement 
group at San Jacinto may have caused them to enroll at a slightly higher rate even before 
experiencing incremental disbursements, the implementation research — which found that 
students often did not fully understand how they would receive their aid — does not point to 
any clear cause of this impact. 

There is one significant impact on credit outcomes in the pooled sample. Program 
group students attempted more credits in the second semester (by 0.3 credits), as indicated in the 
bottom panel of Table 7. However, they did not earn significantly more credits. The impact on 
credits attempted is driven by San Jacinto, where students in the program group attempted about 
one credit more on average than students in the standard group in the second program semester 
(Table 8). This outcome is at least partly explained by the positive impact on second-semester 
enrollment at San Jacinto, which results in fewer students taking zero credits in the program 
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Table 8 


Academic Outcomes by College, Fall 2014 to Spring 2016 Cohorts 



Houston Community College 



San Jacinto College 



Outcome 

Program 

Group 

Standard 

Group 

Difference 

P-Value 

Program 

Group 

Standard 

Group 

Difference 

P-Value 

Subgroup 

Difference 

First program semester 

Enrolled (%) 

96.6 

97.1 

-0.5 

0.441 

97.8 

96.8 

1.0* 

0.085 

f 

Credits attempted 

10.1 

10.1 

0.0 

0.759 

11.4 

11.2 

0.2 

0.158 


Credits earned 

6.6 

6.5 

0.1 

0.714 

8.2 

8.1 

0.0 

0.921 


Sample size (total = 6,264) 

1,485 

1,495 



1,639 

1,645 




Second program semester 

Enrolled (%) 

66.8 

68.2 

-1.4 

0.422 

72.6 

67.0 

5 7 *** 

0.004 

ttt 

Credits attempted 

6.5 

6.6 

-0.1 

0.715 

8.8 

8.0 

Q g**sjc 

0.008 

tt 

Credits earned 

4.4 

4.6 

-0.2 

0.430 

6.3 

6.0 

0.3 

0.264 


Sample size (total = 4,220) 

1,125 

1,132 



980 

983 





SOURCE: MDRC calculations using transcript data from Houston Community College and San Jacinto College. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

Estimates are adjusted by cohort. 

The first program semesters for the three student cohorts were fall 2014, spring 2015, and fall 2015 at San Jacinto College and spring 2015, fall 2015, 
and spring 2016 at Houston Community College. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; 
* = 10 percent. 

A two-tailed t-test was applied to differences in impacts between subgroups. Statistical significance levels are indicated as: fff = 1 percent; ff = 5 
percent; f = 10 percent. 



group. While program group students also earned slightly more credits, that difference is not 
statistically significant. The theory of change hypothesized that there would be impacts on 
credits earned; this will be tested again with the full sample in the final report. 

Subgroup Analyses 

In addition to the impact comparisons by college, several prespecified exploratory sub¬ 
groups were analyzed. Generally speaking, the patterns of impacts observed did not offer 
meaningful insight into groups of students who were more or less likely to benefit from the 
intervention. 

Subgroups explored included students who identified on the FAFSA as financially in¬ 
dependent; students who expected to take out loans; students whose Expected Family Contribu¬ 
tion (an estimate of the student’s ability to pay for college) was zero; students under 20 years 
old; students required to take developmental courses; and students enrolled full time at the point 
of random assigmnent. Additionally, the impacts of Aid Like A Paycheck were examined and 
compared based on students’ gender, racial, and ethnic identification. In most cases, there were 
no differences in the effectiveness of the program based on these subgroups. In cases where 
differences existed, the findings generally correlated with the differences between impacts at the 
two colleges. For example, the estimated impact of biweekly disbursements on persistence was 
larger for Hispanic students than for non-Hispanic students; however, this difference may 
simply arise from the fact that San Jacinto — where impacts on persistence were observed — 
had a larger Hispanic population than HCC. 

These subgroups will be analyzed again in the final report. With a larger sample and 
longer follow-up, these analyses may provide evidence of variations in the impacts of biweekly 
disbursements on various groups of students. 


Costs of Aid Like A Paycheck 

• At HCC and San Jacinto, the transition from lump sum to biweekly financial 
aid disbursements required upfront investments to revise automated systems, 
to review and adjust calculations for additional aid or other changes, to train 
staff members, and to manage other aspects of the change. 

• At this point there is little evidence of substantial cost savings to either the col¬ 
leges or the federal government from the policy of biweekly disbursements. 
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Upfront Investment 


Financial aid disbursement processes are fundamentally complex, and scheduling bi¬ 
weekly refund disbursements throughout the semester added to this complexity. Both colleges 
revised their automated systems to facilitate biweekly disbursements, and San Jacinto and HCC 
each spent about $200,000 to program, test, and execute new automated disbursement rules, 
loan repackaging procedures, and other processes to distribute biweekly disbursements. San 
Jacinto hired outside infonnation technology consultants for much of their initial systems 
revision, while HCC used the resources of in-house infonnation technology personnel. Addi¬ 
tionally, some outside consultation was provided at no charge; at HCC, representatives from 
PeopleSoft, an administrative software system, donated time to help with the conversion. 

The transition to biweekly disbursements also entailed time commitments from a num¬ 
ber of college staff members. For example, financial aid advisers and business office staff 
members were trained in the new process, senior administrators oversaw and guided the 
changes, and financial aid advisers reported spending time answering student questions about 
the new biweekly disbursement policy. 

Requirements of the Randomized Controlled Trial 

As required for this random assignment evaluation, each college maintained two dis¬ 
tinct systems of disbursement. Staff members at San Jacinto, in particular, attributed some of the 
challenges of automating the biweekly disbursements to the fact that they needed to maintain 
separate systems. Maintaining both biweekly and lump sum disbursement rules may also have 
created confusion among the staff and students, thereby increasing the amount of staff time and 
resources required to address student concerns. On the other hand, having just one system and 
set of rules — and providing incremental disbursements to more students — could require 
additional resources, such as staff time in financial aid and business offices to check disburse¬ 
ments, generate refunds, and address student questions. 34 

In order to better understand how the costs might differ when biweekly disbursements 
are universally adopted, in the absence of an experiment such as Aid Like A Paycheck, 
MDRC is observing how West Hills Community College in California is implementing the 
biweekly disbursement policy district-wide. To date, it is too early to understand how their 
costs compare with those of the two colleges in this study, although it appears that the cost of 


34 At San Jacinto, where the Banner system is used, staff members predicted that shifting to incremental 
disbursements as their single method might significantly reduce their ongoing operating costs. At HCC, where 
the PeopleSoft system is used, staff members predicted that shifting to biweekly disbursements for all might 
increase their ongoing costs due to the amount of staff time required to check calculations, adjust disburse¬ 
ments, and communicate with students about their refunds. 
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adopting biweekly disbursements may decrease when a college adopts the new approach for 
all students receiving financial aid refunds. 

Do Biweekly Disbursements Save Colleges Money? 

Aid Like A Paycheck’s theory of change identified reductions in R2T4 as the outcome 
with the most immediate potential for college savings. In the first program semester, there is no 
statistically significant impact on the colleges’ pooled average R2T4 obligation. However, it is 
worth noting that there was a statistically significant reduction in the amount of outstanding debt 
owed by students to the college (driven by impacts at San Jacinto), as well as a reduction in 
R2T4 at HCC in the first semester. These reductions can be seen as a positive change for 
colleges, as well as for students. 

Could Biweekly Disbursements Save the Government Money? 

Biweekly disbursements may also change student outcomes in ways that affect the total 
amount of federal grants, state grants, and loans to students. Given the lack of substantial 
impacts on overall financial outcomes at this point in the study, there is little evidence of 
substantial cost impacts for either the state or the federal government. 35 The final report in 2018 
will include longer follow-up on a larger sample of students to better understand how biweekly 
disbursements financially affect students, colleges, and the government. 


Conclusions 

Many students struggle financially, and although biweekly disbursements of financial aid 
cannot change the fact that existing aid often does not cover the full cost of attendance, this 
policy may still provide academic and financial benefits to students. Colleges with a significant 
number of students who receive refunds may be most likely to consider this policy, and it may 
also be particularly appealing for colleges wishing to be transparent in aligning the awarding of 
aid with continued enrollment, in order to signal to students and others that financial aid refunds 
are intended to support students’ expenses while in school. 

Lessons at the Midpoint of the Study 

At this point in the evaluation, the findings on the results of the Aid Like A Paycheck 
policy are mixed. Academically, students at San Jacinto benefited on average from the program, 


35 Disbursing refunds biweekly, rather than in a lump sum early in the term, can produce net interest sav¬ 
ings for providers of financial aid (whether Pell Grants, state aid, or other aid). During the time of this study, 
however, real interest rates were at historically low levels and any interest savings would be insubstantial. 
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and the 6 percentage point increase in their persistence at the college is particularly promising. 
However, these academic benefits were not evident at HCC in the first few semesters of the 
evaluation. Longer follow-up, combined with additional semesters of study intake and a larger 
number of students in the study, may expose additional academic impacts of the program or 
may shed light on possible reasons that biweekly refunds were observed to have a greater 
academic impact at one college than at the other. 

From a financial health perspective, the results are also mixed; students assigned to re¬ 
ceive biweekly refunds appear to have experienced increased financial stress, but also some 
financial benefits, from the Aid Like A Paycheck program. As indicated by their survey 
responses, students assigned to receive biweekly disbursements were more likely than those 
assigned to the standard group to feel financial stress and worries early in the semester. They 
also often expressed uncertainty about the amount and timing of their aid, and this may have 
contributed to their financial stress. Colleges that consider shifting to a policy of incremental 
disbursements may want to carefully and intentionally structure their messaging and 
communications about the policy in order to increase understanding and knowledge of the 
program by students. Better understanding may help students to plan financially and may 
mitigate some of the anxiety that students experienced as a result of the program at the 
Houston colleges. 

On a more positive note, measures of financial aid disbursed and debt accumulated 
suggest that incremental disbursements do not hurt, and in some cases help, students finan¬ 
cially. Notably, biweekly disbursements reduced students’ debt to the college and reduced the 
use of federal loans. Researchers have noted that community college students nationally take 
out less money in loans per year than students at four-year institutions, but that loan debt can 
still be an important factor for some of these students. 36 For colleges striving to reduce the 
portion of their students who take out loans, biweekly disbursements may be a useful 
tool to achieve that goal while ensuring that students who stay enrolled are not restrict¬ 
ed from borrowing. 

Even under a biweekly disbursement policy, however, providing students with more aid 
earlier in the term may increase the risk of debt from overpayments. For example, San Jacinto 
disbursed the first biweekly refund a week before the standard lump sum payment. Qualitative 
data suggest that this early payment may have been appreciated by students who needed those 
funds early in the term, but it may partially explain why students in the program at San Jacinto 
did not experience the R2T4 reductions that were seen at HCC. In a similar vein, HCC dis¬ 
bursed aid to students in the program based on their expected enrollment; thus some students 
received refund amounts based on full-time enrollment before their late term or part-of-tenn 

36 Baum (2016); Barrett and Laitinen (2017). 


35 



classes had started. If those students withdrew from college or reduced their enrollment status to 
part-time, they may have had to pay back some of these funds. This rule may partially explain 
why students in the program at HCC did not experience an overall reduction in debt to the 
college, despite their reduction in R2T4 debt. Within the framework of incremental dis¬ 
bursements, policies and rules that provide students with more aid earlier in the term may 
increase the risk of overpayments and student debt to the college, but these decisions can 
be balanced with others in order to provide overall reductions in student debt. 

These implementation decisions affect not only students’ finances, but colleges’ budg¬ 
ets as well. Although administrators at the colleges in this study are testing biweekly disburse¬ 
ments with the primary goal of helping students, they are also concerned about the impacts of 
the policy on their own budgets. Reductions in R2T4 were expected to be a key part of potential 
college savings, but these interim findings show that not only are R2T4 impacts less evident 
than expected, but the overall magnitude of R2T4 is smaller than expected for students in the 
standard as well as the program group. In fact, administrators at both colleges have recently 
reported notable college-wide reductions in R2T4 over the past few years, unrelated to the shift 
to biweekly disbursements for this study. This reduction in the base amount of R2T4 leaves less 
room (and perhaps less need) for policies to affect the amount of funds required to be returned 
and may make the topic somewhat less relevant for college budgets. Regardless, even small 
impacts or costs can add up in the long run, and R2T4 remains an important matter for the 
students who are penalized. Fewer than 5 percent of students in this study had R2T4 penalties, 
but for those penalized, average amounts were well over $500. To put this in context, in a recent 
survey of community college students over half the respondents stated that they would not be 
able to come up with more than $500 if an unexpected need arose. 37 The potential for R2T4 
savings for colleges may not be as large as imagined, but biweekly disbursements may 
lead to reductions in R2T4 (as well as student debt to the college) that are meaningful to 
students. 

Remaining Questions and Next Steps 

The research to date is inconclusive on the academic benefits of biweekly disburse¬ 
ments, and there is no clear explanation why students at one college were observed to benefit 
more than students at the other. These differences in academic impacts — as well as some of 
the differences in financial impacts — may be a result of the unique student characteristics at 
each college, of the policy’s implementation differences at the colleges, or of other factors. As 
long as these differences are unexplained, other colleges should look at these findings with 
some caution, knowing that their individual results may vary as well. 


37 Center for Community College Student Engagement (2017). 
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Additionally, because the biweekly refund policy was new and was implemented only 
for a randomly selected minority of students, neither college’s experience provides an indication 
of what a fully implemented policy of incremental disbursements might look like, or whether 
this policy could substantially change the way that financial aid is viewed by students and 
others. Leaders at the colleges have discussed ways in which broader messaging to students 
could potentially be achieved and have recently begun to incorporate discussions of financial 
aid disbursement policies into some staff and faculty training sessions. By sharing this infor¬ 
mation with staff and faculty, they hope to expand knowledge of the relationship between 
student enrollment and financial aid and perhaps increase understanding of students’ financial 
circumstances as well. 

In 2018, MDRC will release a final report examining the longer-term impacts of bi¬ 
weekly disbursements on students’ financial and academic outcomes at HCC and San Jacinto, 
with up to three additional semesters of follow-up and over 2,500 additional students in the 
sample. The final report will also further explore the costs and potential savings to colleges and 
the government. These additional quantitative data and analyses may help answer some of the 
outstanding questions noted above. The final report will also include qualitative research on the 
program and students at West Hills, where biweekly disbursements are now the norm for 
students college-wide. The experiences at West Hills may shed light on how a college’s 
students, faculty, and staff respond to the more pervasive messaging allowed by universal 
adoption of the biweekly disbursement schedule. 
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About MDRC 


MDRC is a nonprofit, nonpartisan social and education policy research organization dedicated 
to learning what works to improve the well-being of low-income people. Through its research 
and the active communication of its findings, MDRC seeks to enhance the effectiveness of so¬ 
cial and education policies and programs. 

Founded in 1974 and located in New York; Oakland, California; Washington, DC; and Los 
Angeles, MDRC is best known for mounting rigorous, large-scale, real-world tests of new and 
existing policies and programs. Its projects are a mix of demonstrations (field tests of promising 
new program approaches) and evaluations of ongoing government and community initiatives. 
MDRC’s staff members bring an unusual combination of research and organizational experi¬ 
ence to their work, providing expertise on the latest in qualitative and quantitative methods and 
on program design, development, implementation, and management. MDRC seeks to leam not 
just whether a program is effective but also how and why the program’s effects occur. In addi¬ 
tion, it tries to place each project’s findings in the broader context of related research — in order 
to build knowledge about what works across the social and education policy fields. MDRC’s 
findings, lessons, and best practices are proactively shared with a broad audience in the policy 
and practitioner community as well as with the general public and the media. 

Over the years, MDRC has brought its unique approach to an ever-growing range of policy 
areas and target populations. Once known primarily for evaluations of state welfare-to-work 
programs, today MDRC is also studying public school refonns, employment programs for ex¬ 
offenders and people with disabilities, and programs to help low-income students succeed in 
college. MDRC’s projects are organized into five areas: 

• Promoting Family Well-Being and Children’s Development 

• Improving Public Education 

• Raising Academic Achievement and Persistence in College 

• Supporting Low-Wage Workers and Communities 

• Overcoming Barriers to Employment 

Working in almost every state, all of the nation’s largest cities, and Canada and the United 
Kingdom, MDRC conducts its projects in partnership with national, state, and local govern¬ 
ments, public school systems, community organizations, and numerous private philanthropies. 



